1. Field of Invention
This invention relates to selecting objects on an electronic media.
2. Description of Related Art
One of the major potential benefits of reading electronic media, such as an electronic document, is that readers can select and copy material automatically, without needing to retype it. The readers copy text, graphics and the like for various reasons, including adding quotations to documents that they are writing, or reducing their reading into snippets that they can organize and analyze.
When establishing the authority of a statement, such as in legal writing, quoting is particularly important. That is, legal arguments generally rest on points of law taken from previous cases. Furthermore, quotes are often discontiguous. For readability, legal writers often shorten quotes by using ellipsis. Legal writers also often stitch together sections of separate quotes with clauses of their own.
In addition, digital ink has become available to make annotations on electronic media. For example, the user may electronically underline and/or highlight text and/or add notes in margin regions of electronic documents. However, when the user uses digital ink, such as freeform digital ink, to make notes, the notes often must be retyped or individually copied to another medium if the annotated media needs to be copied to another form of media. This can be extremely time consuming and not accurate.
Discontiguous selection is a common feature of graphical user interface (GUI) systems. For example, in a graphical computer operating system, such as Microsoft Windows®, typically the user needs to use a modifier, such as depressing the control key, in addition to using a mouse button, to add to an existing selection, rather than starting a new selection.
A freeform digital ink whiteboard, known as Tivoli and disclosed in U.S. Pat. No. 5,471,578 to Moran et al., supports freeform ink note taking and annotating as well as selection. In this system, when the user circles an object, the system interprets the circle as a selection gesture rather than an annotation. In this system, explicit interaction of the marks with respect to the underlying object begins when the user completes the circle gesture.
In “Implicit Structures for Pen-Based Systems Within a Freeform Interaction Paradigm”, Proceedings of CHI '95, AMC Press, pp. 487-494, Moran et al. disclose this system and teach selection gestures using vertical or horizontal brackets or pairs of brackets. Using this system, the user can draw a vertical or horizontal bracket or a pair of the brackets to select an underlying object. Similarly, U.S. Pat. No. 5,889,523 to Wilcox et al. discloses a system that selects gestures by underlines.
In these references, the shift from annotation to explicit interaction takes place when the user double taps on a stroke. The system then recognizes the double tap as a gesture. However, these systems are not capable of interpreting more than one annotation as a selection. Although Moran's system does allow users to modify selection gestures with further gestures, these further gestures are explicit modifiers to the selection rather than annotations.
Moran's system and U.S. Pat. No. 5,861,886 to Kurtenbach et al. disclose another relevant selection technique called “freeform enclosure.” In this technique, the system interprets strokes with the endpoints near each other as persistent groupings. The user can tap on the enclosure stroke to select the enclosure and the contained objects. However, in this technique, annotations are not grouped to select underlying objects. Although Moran's system can detect the overlap of two enclosures, and certain gestures within the overlap act on all objects present in the two overlapping enclosures, there is no way for users to select the objects within the intersection of two regions. Furthermore, overlapping is a very limited form of grouping. That is, grouping by overlapping does not allow disparate selections. Moreover, overlapping is not an appropriate way to group freeform underlines, freeform highlights, or text range annotations. Finally, Moran's system acts on the objects that are contained by both enclosures.
Chiu and Wilcox describe, in “A Dynamic Grouping Technique for Ink and Audio Notes”, Proceeding of UIST '98 (Nov. 1-4, 1998), ACM Press, pp. 195-202, a system for freeform ink and audio note taking that determines groups of ink strokes and that allows the user to select a group of strokes and play the associated audio.
U.S. Pat. No. 6,687,876, incorporated herein by reference in its entirety, suggests several types of automatic actions that can take place on annotated text. For example, the system automatically resolve and fetch secondary references when the user annotates a reference. In one mode, the system takes action automatically, as with resolving and fetching references. In another mode, the user invokes the action on one or more documents. In response, the system acts on all of the annotations in those documents. Therefore, the system disclosed in the incorporated 962 application has no concept of selection on text. The incorporated 962 application does not suggest allowing the user to take action on any specific annotated passage.
The interface disclosed in “The Pen-Ivory Project: Exploring User-Interface Design for the Selection of Items from Large Controlled Vocabularies of Medicine”, JAMIA 1996; 3:168-183, allows users to identify a medical condition as “present” or “not present” by circling the medical condition or by crossing out the medical condition, respectively. In the Pen-Ivory system, the stroke is treated as both the selection and the command, rather than treating the selection and command as individual actions.